Skip to main content

Fairness, accountability and transparency in Machine Learning? Jo Bates reports back from ACM FAT* in Atlanta, USA


A couple of weeks ago I travelled to Atlanta, USA to attend ACM FAT* - an interdisciplinary conference that addresses issues of Fairness, Accountability and Transparency in Machine Learning. Officially, I was there on the hunt for potential papers and authors to invite to submit their work to Online Information Review. However, the FAT* field is also closely related to my research interests around the politics of data and algorithms, and my teaching on the Information School’s MSc Data Science. I was keen to check out what was happening in the FAT* community, and feed my findings back into my teaching and into two new projects I am working on in this field: CYCAT & supervising a new PhD student – Ruth Beresford – whose research will investigate algorithmic bias in collaboration with the Department for Work and Pensions.

I was privileged to hear a number of great papers – the best of which engaged critically with issues of social context and justice. My two favourite papers which I highly recommend for anyone interested in these topics were:

Fairness and Abstraction in Sociotechnical Systems (Selbst et al, 2019) was a great paper that spoke to a sense of unease I have felt recently about the increasing amount of technical work attempting to solve the ‘algorithmic bias problem’. The paper – written by an interdisciplinary team of social and computer scientists - not only speaks to such concerns in an engaging and insightful way, but also offers a strong analytical framework that illuminates five “traps” that such technically-driven work often falls into:
  •          Framing: The authors begin by critiquing the ‘algorithmic framing’ common in data science. In such an abstraction, the focus of the data/computer scientist is simply on evaluating, for example, whether the model has high accuracy. They point out that such a framing is ineffective for addressing issues of bias and fairness. Expanding this algorithmic framing to a ‘data frame’ which also involves interrogating directly the data inputs and outputs for issues of bias and fairness, can address some of these issues, but also has its limitations. Instead they advocate data scientists adopting a socio-technical framing which explicitly recognises that any ML model is part of a socio-technical system – and we need to move all the decisions made by humans and human institutions into the abstraction boundary. I couldn’t agree more!
  •         Formalism: This is an important ‘trap’ for computer scientists and mathematicians to be aware of. It relates to the failure to account for the complexity of social concepts such as fairness, bias etc. The meaning of such concepts is contextual and contestable – they cannot be reduced to mathematical formalisms!
  •         The Ripple Effect: This ‘trap’ points to the lack of awareness that when technical solutions are embedded into existing social systems, they can impact upon the behaviours and values of those in the social system – often in unexpected ways.
  •         Portability: This ‘trap’ relates to the problems inherent in repurposing algorithmic solutions from one social context to another – and the inevitable problems of inaccuracy, misleading results, and potential for harm.
  •         Solutionism: And, finally - the simple observation that technologists often fail to recognise that the best solution to a problem may not involve any technology!

Putting some of these ideas into practice was my second favourite paper – which also won the prize for best technical and interdisciplinary paper - Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments (Green and Chen, 2019).

With a focus on risk assessments being used in the US criminal justice system, the authors argue that given risk assessment tools do not actually make decisions, but are used to inform judges’ decisions, it is important to understand how people actually interpret and use the outputs of these tools. Their study is based on an experiment involving Amazon Mechanical Turk workers – rather than actual judges – however, their findings are concerning. They found their participants under-performed the risk assessment tool even when presented with the prediction of the tool; they were unable to effectively evaluate the accuracy of their own decisions or those of the tool; and, most concerning they exhibited biased interaction with the tool’s prediction whereby use of risk assessments in decision making led to participants making higher risk predictions for black defendants and lower risk predictions for white defendants. Clearly, these findings need examining ‘in the wild’, but they are concerning, and evidence the importance of a socio-technical framing as called for by Selbst et al.

While there were some excellent papers presented at FAT*, there were also a good few that fell into some of the ‘traps’ of abstraction identified in Selbst et al’s paper – and this resulted in some interesting commentary about the nature and direction of the field. For example, important questions were raised by Stanford PhD student Pratyusha Ria Kalluri, who drew upon Catherine D'Ignazio and Lauren Klein’s forthcoming book Data Feminism, to question the language of fairness, accountability and transparency, and how it relates to notions of justice. Her comments received a lot of support from attendees and online:


Pratyusha’s observations reflect many of my own – and others in the Critical Data Studies space - concerns about what it means to work across disciplinary boundaries in this field, and the politics of engaging in such work with people who may have very different agendas, assumptions, and understandings about what is at stake. It can sometimes be difficult to know how best to navigate these tensions – but it feels like 2019 could be an important moment for shaping the direction of the field.

Comments

Popular posts from this blog

Raspberry Pi Weather Project now live

A project to create a raspberry pi weather station is currently live in the Information School.  The Sheffield Pi weather station has been created by Romilly Close, undergraduate Aerospace Engineering student at the University of Sheffield.  The project was funded by the Sheffield Undergraduate Research Experience (SURE) scheme and is being supervised by Dr Jo Bates, Paula Goodale and Fred Sonnenwald from the Information School. Information about the Sheffield Pi station and how to create your own can be found on the project website .  You can also see live data from the Sheffield Pi station on Plot.ly , and further information can also be found on the Met Office Weather Observations Website .    This work compliments the School’s existing project entitled ‘The Secret Life of a Weather Datum’ which explores socio-cultural influences on weather data.  This project is funded under the AHRC’s Digital Transformations Big Data call.  It aims to pilot a new approach to im

Our Chemoinformatics Group wins Jason Farradane Award

The Information School's Chemoinformatics Research Group has been awarded the 2012 UKeiG Jason Farradane Award , in recognition of its outstanding 40 year contribution to the information field. The prize is awarded to the three current members of the group,  Professor Val Gillet , Dr John Holliday and Professor Peter Willett . The judges recognised the Group's status as one of the world's leading centres of chemoinformatics research, a major contributor to the field of information science, and an exemplar in raising the profile of the information profession. The School has a long association with the Farradane prize. Its second recipient was long time member of staff Professor Mike Lynch in 1980.

Professor Mike Thelwall gives inaugural lecture

Professor of Data Science Mike Thelwall recently gave his inaugural lecture at the University of Sheffield, entitled  How helpful are AI and bibliometrics for assessing the quality of academic research? The lecture, delivered in the University's Diamond building, was introduced by Head of the Information School Professor Briony Birdi. It covered Mike's research into whether Artificial Intelligence can inform - or replace - expert peer review in the journal article publication process and what this could look like, as well as to what extent bibliometrics and citation statistics can play a role in assessing the quality of a piece of research. Mike also discussed whether tools like ChatGPT can accurately detect research quality. The inaugural lecture was well attended by colleagues from around the University.